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ABSTRACT 

In  many  applications,  it  is  required  that  heterogeneous  multi¬ 
robots  are  grouped  to  work  on  multi-targets  simultaneously. 
Therefore,  this  paper  proposes  a  control  method  for  a  single¬ 
master  multi-slave  (SMMS)  teleoperator  to  cooperatively  con¬ 
trol  a  team  of  mobile  robots  for  a  multi-target  mission.  The  ma¬ 
jor  components  of  the  proposed  control  method  are  the  compen¬ 
sation  for  contact  forces,  modified  potential  field  based  leader- 
follower  formation,  and  robot- task- target  pairing  method. 

The  robot-task-target  paring  method  is  derived  from  the 
proven  auction  algorithm  for  a  single  target  and  is  extended 
for  multi-robot  multi-target  cases,  which  optimizes  effect-based 
robot-task-target  pairs  based  on  heuristic  and  sensory  data.  The 
robot-task-target  pairing  method  can  produce  a  weighted  attack 
guidance  table  (WAGT),  which  contains  benefits  of  different 
robot- task- target  pairs. 

With  the  robot-task-target  pairing  method,  subteams  are 
formed  by  paired  robots.  The  subteams  perform  their  own  paired 
tasks  on  assigned  targets  in  the  modified  potential  field  based 
leader-follower  formation  while  avoiding  sensed  obstacles.  Sim¬ 
ulation  studies  illustrate  system  efficacy  with  the  proposed  con¬ 
trol  method. 

INTRODUCTION 

Cooperative  control  of  multi-robotic  systems  has  been  stud¬ 
ied  extensively  in  recent  years  [5,7, 8, 11-13, 15-17],  especially 
for  some  tasks  that  cannot  be  handled  by  one  single  robot.  It 
can  improve  dexterity  of  robots  and  enlarge  application  fields  of 
robots.  Thus,  many  cooperative  control  algorithms  have  been 
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proposed  so  far  [5, 7,  8, 11-13, 15-17].  There  are  two  types  of 
cooperation.  One  is  the  cooperation  without  force  interactions 
among  robots  (unconstrained  motion  tasks)  and  the  other  is  with 
them  (constrained  motion  tasks).  In  the  former  type  of  cooper¬ 
ation,  task  planning  is  one  of  the  main  technical  problems,  but 
the  same  positional  controller  as  that  of  single  robot  can  be  used 
and  it  can  be  realized  very  easily.  Therefore,  this  type  of  co¬ 
operation  has  been  practically  used  for  target  captures  or  enclo¬ 
sure  [5, 12, 15, 17].  In  the  latter  type  of  cooperation,  under  the 
interactions  of  forces,  design  of  the  control  strategies,  which  can 
keep  inner  forces  between  robots  to  be  desired  values  and  also 
ensure  the  stability  of  the  controllers,  becomes  the  most  criti¬ 
cal  problem.  This  has  been  seen  for  target  transportations  as  the 
force  or  impedance  controller  has  been  commonly  used  [7, 8, 16]. 
Besides  these  two  types  of  cooperation,  the  transition  between 
them  has  been  investigated  in  some  papers  [11,  13].  The  tran¬ 
sition  involves  a  smooth,  stable  switching  between  motion  and 
force  control  when  instability  and  large  force  spikes  during  the 
switching  are  avoided.  However,  in  those  papers  [11, 13],  a  con¬ 
trol  method  was  not  developed  to  split  a  robot  team  into  several 
sub-teams  to  do  the  different  motion  tasks  when  applications, 
such  as  military  operation,  space  exploration,  and  etc.  request  all 
robots  to  do  the  multi-motion  tasks  simultaneously. 

Furthermore,  in  many  applications,  unstructured  nature  of 
the  worksite  environments  and  the  limitations  of  the  current  sen¬ 
sors  and  computer  decision-making  technologies  prohibit  the  use 
of  fully  autonomous  systems  for  the  operations  [6,8].  Therefore, 
it  is  required  that  the  human  decision  making  be  involved  in  the 
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systems.  Teleoperators,  in  which  a  human  operator  is  an  inte¬ 
gral  part  of  the  control,  are  established  to  integrate  the  human 
decisions  to  the  control  loop  of  the  systems.  By  minimizing  the 
required  human  resources  and  amplifying  the  human  effort,  the 
single-master  multi-slave  (SMMS)  teleoperation  has  been  con¬ 
sidered  in  this  paper. 

Fong  et.  al.  suggested  the  collaborative  control  with  dia¬ 
logue  functions  to  remotely  operate  the  multi-robot  via  a  master 
robot  to  search  in  an  open  area  [5].  With  their  approaches,  the 
slave  robots  have  more  freedom  in  execution  and  are  more  likely 
to  find  a  good  solution  by  themselves  when  they  have  a  prob¬ 
lem.  The  human  operator  is  able  to  function  as  a  resource  for 
the  slave  robots,  providing  information  and  processing  just  like 
other  system  modules.  Nevertheless,  their  framework  for  coordi¬ 
nation  does  not  contain  any  mechanism  for  remotely  regrouping 
the  team  robots  into  several  sub-teams  to  carry  out  multi-tasks 
containing  unconstrained  and  constrained  motions  to  capture  and 
transport  multi-targets  simultaneously. 

Many  different  methods  to  assign  multi-tasks  and  multi¬ 
targets  to  subteams  have  been  widely  applied  in  fully  automatic 
coordinated  multi-robotic  systems  [10].  The  methods  are  a  ge¬ 
netic  or  improved  genetic  algorithm  [3],  ant  colony  system  [9], 
particle  swarm  optimization  [10],  market-based  approaches  [4], 
and  auction  or  decentralized  cooperation  auction  [14].  Nonethe¬ 
less,  they  have  no  ability  to  stably  converge  to  a  global  optimum. 
Therefore,  Bogdanowicz  and  Coleman  et.  al.  recommended  a 
method  for  optimization  of  effect-based  weapon-target  pairings 
[1]  to  decide  a  preferred  weapon-target  combination  for  engag¬ 
ing  a  given  target  by  scanning  attack  guidance  tables.  Different 
from  those  previously  mentioned  methods,  it  is  a  rule  and  func¬ 
tion  based,  not  an  optimized  method.  Therefore,  it  can  converge 
rapidly  and  produce  a  suboptimal  solution  stably.  Nonetheless, 
Bogdanowiczs  and  Colemans  optimization  was  only  focused  on 
matching  several  weapon  combinations  with  numerous  targets  in 
a  single-motion  task,  which  could  not  fit  into  a  general  mission 
with  multi-motion  tasks. 

Due  to  the  above  mentioned  problems,  the  primary  objective 
of  this  paper  is  to  develop  a  control  method  for  a  SMMS  tele¬ 
operator  to  cooperatively  control  a  team  of  heterogeneous  mo¬ 
bile  robots  for  robot- task- target  pairing.  Primary  components 
of  the  proposed  control  method  are  (1)  modified  potential  field 
based  leader-follower  formation,  (2)  compensation  for  contact 
forces  [2],  and  (3)  robot- task-target  pairing.  During  the  oper¬ 
ation,  the  human  operator  only  focuses  on  controlling  a  team 
leader  robot.  All  other  team  robots  autonomously  make  a  for¬ 
mation  regarding  its  positions  and  velocities  based  on  sensory 
information.  Therefore,  the  formation  is  adapted  by  modifying 
their  paths  for  obstacle  avoidance  by  using  the  modified  poten¬ 
tial  field  based  leader- follower  formation  controller  developed  in 
our  research  papers  [2].  Moreover,  in  the  constrained  motion, 
the  compensator  for  the  contact  forces  enables  the  slave  robots 
to  adapt  their  forces  acting  on  the  transported  target  to  have  a 


firm  grip  of  it.  When  the  team  leader  is  close  to  targets,  the 
team  robots  are  assigned  tasks  and  targets  correspondingly  with 
the  robot-task-target  pairings,  and  then  the  subteams  are  formed 
based  on  optimal  robot- task-target  pairs.  In  addition,  the  subteam 
leaders  are  selected  based  on  robot  functionalities  and  proximity 
to  the  targets  to  lead  the  subteams. 

The  rest  of  this  paper  is  organized  as  follows.  In  Section 
2,  the  control  method  that  integrates  the  primary  components  to 
execute  different  motion  tasks  simultaneously  with  different  sub¬ 
teams  for  a  robot- task-target  approach  is  proposed.  In  Section  3, 
the  effectiveness  of  the  task  achievement  of  the  SMMS  system 
with  the  proposed  control  method  were  evaluated  through  simu¬ 
lation  studies.  Section  4  concludes  this  paper  and  shows  future 
research  directions. 

SEMI-AUTONOMOUS  SINGLE-MASTER  MULTI-SLAVE 
(SMMS)  TELEOPERATION  CONTROL  METHOD  FOR  A 
ROBOT-TASK-TARGET  APPROACH 

This  paper  extends  the  preliminary  concepts  of  the  semi- 
autonomous  SMMS  teleoperation  control  method  [2]  which  was 
only  focused  on  a  single-target  into  a  multi-target  operation,  i.e. 
several  simultaneous  target  captures  and  transportations,  in  a 
complicated  environment.  The  major  difference  for  cooperative 
robots  between  completion  of  the  multi-task  and  single-task  is  a 
robot-task-target  pairing  method. 

In  this  paper,  we  develop  the  robot-task-target  pairing 
method  to  make  the  semi-autonomous  SMMS  teleoperation  con¬ 
trol  method  proposed  in  [2]  be  able  to  deal  with  the  multi-task 
on  the  multi-target.  The  detail  of  the  robot-task-target  pairing 
method  mentioned  above  is  formulated  in  the  following  subsec¬ 
tion. 

Robot-Task-Target  Pairing  Method 

Consider  such  a  scenario,  in  a  two-dimensional  and  limited 
rectangular  environment  X  with  nc  square  cells,  np  slave  robots 
pursue  ne  targets,  for  np>ne.  The  set  of  the  robots  is  denoted  by 
a  matrix  of  A  =  [a\,a2,  ••••arip\  where  aj  is  the  jth  robot  matrix. 
The  jth  robot  capability  vector  for  the  tth  task  is  denoted  by  C*, 
1  <j<np,  and  the  set  of  targets  is  expressed  as  a  target  matrix  of 
T  =  [7j ,  72,  ....T^J  where  Tne  is  the  neth  target  matrix.  The  vector 
representing  the  capability  required  to  accomplish  the  tth  task  on 
the  Tth  target  is  denoted  by  CJ ,  1  <  T  <ne.  Agent  AUT  denotes 
the  teams  of  robots  and  targets.  For  simplification,  we  assume 
that  both  space  and  time  can  be  quantized,  therefore  the  envi¬ 
ronment  can  be  regarded  as  a  finite  collection  of  cells,  denoted 

by  Xc  =  1,2, . ,nc.  There  exist  some  static  obstacles  with  fixed 

sizes  and  regular  shapes,  and  their  locations  are  determined  by 
the  mapping  m  :  Xc  — »  0, 1,  for  Vx  G  Xc,M(x )  >  threshl  indicates 
that  the  cell  x  is  occupied  by  obstacles.  Vx  G  Xc,M(x )  <  threshl 
indicates  that  the  cell  x  is  free,  where  threshl  <  threshl  rep¬ 
resents  the  threshold  value  between  0  and  1.  Each  of  the  het- 
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erogeneous  team  robots  needs  different  capabilities  to  complete 
different  tasks  on  different  targets,  such  as  the  target  capture  and 
transportation. 

Robot  Capability 

For  the  tth  task,  jth  robot,  and  1  <  m  <  u,  the  weighted  ca¬ 
pability  vectors  of  the  jth  robot  to  complete  the  Ith  task  can  be 
defined  as 


C)  =  wTjdiag{btjl,btj2, ....,b)u}  [dn  .  c)m]T  (1) 

where  u  is  the  maximum  number  of  the  vectors,  each  of 
which  represents  the  individual  functionality.  All  heterogeneous 
robots  are  represented  by  the  set  of  robot  matrices,  e.g.  A  = 

-  an  an  al3  .  aXr  n 

a.2}..  a.21..  a.13..  a.2!..  where  nv,  for  0  <  nv  <  np ,  is  the  total 

_anv  1  anv2  anv3  .  anvr  _ 

number  of  the  robots  in  the  team,  and  r,  for  0  <  r  <  ne,  is  the 
total  number  of  the  tasks.  cl-k  is  a  capability  vector  for  the  kth 
functionality  and  the  tth  task,  wj  is  a  positive  integer  such  that 
for  the  given  target  T  and  robot  j9  the  following  is  satisfied.  If 
the  robot  is  assigned  to  the  target,  wj  =  0,  otherwise,  wj  =  1. 
The  uxu  dimension  diagonal  matrix  of  bl-u  is  used  to  estimate 
the  percentage  of  possibility  of  using  the  u  x  1  dimensional  ca¬ 
pability  vector  Oj  to  do  the  tth  task  by  the  jth  robot  successfully. 
However,  if  the  jth  robot  does  not  have  the  capability  d-k,  then 
the  b)k  is  0.  Each  robot  matrix  in  A  has  more  than  one  weighted 

capability  vector,  e.g.  for  the  jth  robot  and  tth  task,  ajt  —  [o)]7 . 
Capability  Required  to  Execute  Tasks  on  Targets 

It  is  assumed  that  there  are  p  tasks  which  need  to  be  done 
independently  and  simultaneously.  All  tasks  are  represented  by 
a  set  of  task  matrices  e.g.  {t\ ,  in  the  system  for  p  <  ne , 
i.e.  one  task  can  be  paired  to  two  or  more  targets,  but  each  tar¬ 
get  can  only  be  paired  to  one  task.  The  capability  vector  that  is 
required  to  accomplish  the  tth  task  on  the  Tth  target  is  defined  as 


cj  =diag{pT,P?2,....,tfu}Ctu  (2) 

where  the  u  x  u  dimension  diagonal  matrix  of  /3^  is  used  to  de¬ 
scribe  the  percentage  of  possibility  of  using  the  u  x  1  dimension 
capability  vector  Ctu  with  which  the  robot  can  finish  the  tth  task 

on  the  Tth  target.  Ctu  =  [ct\ . ctu]T  when  the  total  number  of 

the  functionalities  is  u.  ctu  is  the  capability  vector  that  is  required 
to  complete  the  tth  task  with  the  uth  functionality.  However,  if  the 
tth  task  can  not  be  done  successfully  by  any  robot  with  the  capa¬ 
bility  Ctu  on  the  Tth  target,  then  the  j 3fu  is  0.  Otherwise,  is  1. 
Subteam  Capability 

The  subteam  is  a  combination  of  the  multi-robots  that  work 
on  the  tth  task  cooperatively.  For  the  jth  robot,  tth  task,  and 


TABLE  1 :  Weighted  Attack  Guidance  Table  (WAGT) 


Subteam  1 

mN:i 

Subteam  n 

rriN,n 

i 

d1  d  1 

D\ ’  Dxn 

m,n 

B]  v-,B2xl 

m2,i 

t>2  d  2 

nlni  '  •  •  ’ nxn 

m,n 

oN  dN 

oN  r>N 

nl n’  *  •  •  ’nxn 

mN,n 

cijt  0,  —  cijt  for  p  d  b  1  and  dmax  d  d  d  dmin,  cimin  ^  1, 

and  amax  <  np  where  np /  ( amax  —  amin  +  1)  —ns  where  ns  is  the 
total  number  of  subteams.  The  yth  subteam  is  represented  by  the 


U(amin ’b  .  U(amaxA) 

matrix  of  Dy  '  U(amin  ■  2)  I!!;;!  u^am.ax:2^  ,  because  the  robot  team  de- 

_  U(aminA  .  U{amax,r)  _ 


noted  by  A  can  be  formed  by  several  subteams,  one  of  which  is 
denoted  by  Dy,  i.e.  A  =  {£>i,Z)2,  -,Dy,  ..,Dq}  where  q  is  the  total 
number  of  the  combinations  of  multi-robots  (robot  subteams)  in 
the  robot  team.  For  the  jth  robot  and  tth  task,  if  C-  >  0,  then 


Q(ia,t)  =  C)  for  np  >ia>  1  (3) 

where  the  Q  =  [Q( i,o  •••  Q{npp]  is  a  positive  integer.  The  yth  sub¬ 
team  capability  vector  for  the  tth  task  is  defined  as 


ia=ya 

=  E  (4) 

ia=yb 

where  y^  —  ya,  >  ya,  is  the  total  number  of  the  robots  in  the  yth 
subteam.  ya  is  the  first  and  y ^  is  the  last  indices  of  the  elements  in 
the  matrix  for  the  given  task  t  and  the  yth  subteam.  The  yth 
subteam  is  able  to  perform  the  tth  task  on  the  Tth  target  if  the  con¬ 
dition,  Cl  <Cy,  ,  is  satisfied.  The  subteam  leader  is  selected 

when  its  magnitude  of  the  capability  vector  Cj  is  largest  among 
the  others  in  the  same  subteam.  The  subteam  leader  knows  all 
capability  information  about  its  subteam  members. 

Bidding  Winner  Determination 

In  Table  1 ,  is  the  positive  integer  weight  for  the  nth  sub¬ 
team  to  bid  on  the  xth  task  and  Nth  target.  If  CJ  x  is  smaller 

&  {ya-ybd 

than  the  base  price  which  is  a  positive  integer,  or  the  Nth  tar¬ 
get  has  already  been  assigned  to  the  nth  robot  subteam,  m ^  is 
0.  Otherwise,  is  1.  By  arranging  m^)Yl  and  B^n  into  Table 
1,  called  Weighted  Attack  Guidance  Table  (WAGT),  each  row 
of  WAGT  corresponds  to  a  target  with  Tasks  (1  to  x)  and  Robot 
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Subteam  (1  to  n)  when  v  is  the  total  number  of  the  tasks,  and  n 
is  the  total  number  of  the  subteams  formed  in  the  team.  In  ad¬ 
dition,  each  column  of  WRT  corresponds  to  a  robot  combination 
(Robot  Subteam)  that  accomplishes  Tasks  (1  to  x)  on  Targets  (1 
to  N)  when  N  is  the  total  number  of  the  targets.  Therefore,  there 
are  the  N  rows  and  n  columns  in  WRT.  The  scanning  proceeds 
from  the  first  to  the  last  column.  Hence,  the  robot  combination 
(Robot  Subteam)  specified  in  column  i  takes  precedence  over 
combination  of  robots  specified  in  column  i  +  1.  From  Table  1, 
for  the  rth  subteam  and  Nth  target,  the  subteam  bid  matrix  can 

be  formed,  i.e.  B(N,n)  =  [B^n  B%n  B^n . B%n].  The  maximum 

value  of  the  element  in  the  matrix  of  B(N,n)  is  the  bid  value  for 
the  task  which  is  preferred  to  be  done  on  the  Nth  target  by  the  nth 
subteam.  For  example,  for  the  yth  subteam  and  kth  target,  the  bid 
value  is  weighted  as  follows. 


B(k,y)  =  -X*))  (5) 

t=  1 

where  Xky  is  the  positive  integer  weight  for  the  yth  subteam  to 
do  the  kth  target.  If  the  tth  task  is  the  most  preferred  by  the 
yth  subteam  to  be  done  on  the  kth  target  when  Bky  is  the  maxi¬ 
mum  value  of  the  element  in  the  matrix  of  B(k,y),  then  Xtky  =  0. 
Otherwise,  Xky  =  1.  The  target  bid  matrix  can  be  created,  i.e. 

Bk  =  . B(k,n)]  for  the  kth  target.  Therefore,  based  on 

the  given  subteams,  targets,  tasks,  WAGT,  and  optimization  of 
the  robot-task-target  pairing  that  is  described  below,  the  bidding 
winner  determination  is  made. 

The  optimization  of  the  robot-task-target  pairing  is  formu¬ 
lated  as  follows.  Given  the  robot  subteam  y,  targets  T ,  tasks 
t ,  and  WAGT,  an  assignment  of  the  subteam  is  found  in  such  a 
format  that  WAGT  is  satisfied,  and  its  corresponding  objective 
function  in  Eq.  (6)  is  maximized  within  the  given  constraints  in 
Eq.  (7).  Therefore,  we  can  state  the  optimization  problem  as 
follows.  For  Target  k  and  Subteam  1  —  n  as  seen  in  Table  1,  the 


objective  function  is  ObjFun(k )  =  [(£(&, i>v) .  ] 

maximize  ObjFun(k )  (6) 


Subject  to 


y^B{k,y)>  0  (7) 

y=i 


where  m^y  is  the  positive  integer  weight  for  the  yth  subteam  and 
the  kth  target.  Initially,  all  m^y  is  equal  to  one  if  no  subteam  is 
assigned  to  any  target.  However,  if  Subteam  S  is  assigned  to  Tar¬ 
get  T ,  mshr  is  equal  to  zero  V  Si  ^  S.  Hence,  Subteam  S  that 
proposes  the  maximum  affordable  value  (B(T,S)mT,s)  can  win 
Target  T  by  solving  Eqs  (6)  within  the  constraints  Eq.  (7).  By  us¬ 
ing  the  robot-task-target  pairing  method,  the  subteam/task/target 
pairs  are  stored  into  the  resulted  matrices  e.g.  the  pair  matrices 
and  given  WAGT.  In  order  to  split  its  team  into  some  subteams 
to  execute  different  tasks  on  different  targets  simultaneously,  our 
proposed  control  method  in  [2]  is  modified  into  the  system  with 
the  robot-task-target  pairing  method.  The  robot-task-target  pair¬ 
ing  method  is  created  to  enable  the  system  based  on  the  found 
pairs  to  form  subteams,  appoint  the  robots  as  a  subteam  leader 
and  followers,  pair  the  tasks  to  the  subteams,  and  generate  the 
position  and  force  reference  inputs  to  the  subteams  to  work  on 
the  given  targets.  The  other  components  of  the  proposed  control 
method,  e.g.  the  modified  potential  field  based  leader-follower 
formation  and  the  contact  force  compensators,  are  similar  to 
those  in  [2]. 

SMMS  Teleoperator  with  the  Proposed  Modifications 

The  SMMS  teleoperator  with  integrating  the  above  men¬ 
tioned  control  methods  are  modified  into  Figure  1.  Figure  1 
represents  the  overall  architecture  of  the  modified  teleoperation 
system.  The  master  and  slave  subsystems  in  Figures  la  and  lb, 
respectively,  are  connected  over  the  wireless  internet.  The  master 
subsystem  is  the  same  as  the  one  in  our  papers  [2].  The  difference 
from  the  one  in  [2]  is  that  the  slave  subsystem  with  the  proposed 
control  methods  is  operated  fully  autonomously  for  two  reasons. 
(1)  Human  commands  via  the  master  subsystem  are  temporarily 
not  available  due  to  intermittently  disrupted  or  delayed  transmis¬ 
sion  between  the  subsystems.  (2)  The  team  formed  by  the  slave 
robots  is  divided  into  the  subteams  to  simultaneously  perform 
the  task  on  the  target  when  the  subteam  robots  are  successfully 
paired  to  the  proper  tasks  and  targets  with  the  robot-task-target 
pairing  method.  The  modified  system  shown  in  Figure  1  is  for¬ 
mulated  into  the  following  equations  of  motion. 

Master: 

Mmem  T~  Kmem  —  0  (8) 

ith  Slave: 

Msiesi  +  Bsiesi  +  Ksiesi  =  UT  +  U0  +  (l-a)(l-X)Uf  (9) 

+CeSFsi 

where  Uf  is  the  virtual  bonding  between  robots.  Uj  is  the  vir¬ 
tual  attraction  to  the  target  while  UQ  is  the  virtual  repulsion  from 
the  obstacles.  Ce  is  the  force  compensator  to  regulate  the  contact 
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TABLE  2:  SMMS  simulations  for  a  multi-target  mission 


Human 

Operator 


Matter  A  Shsc  ftwioun*  * 
Leadership  A  Target  McntifitJtiom 


Distributed  Slave  Sub-System 
(Group  Subgroup  Leader  & 
Followers) 


Sims 

Robot  Types 

Control  Objectives 

Sim  (1) 

Homogeneous 

Robots 

Non-Robot-Task-Target  Pairing 

Sim  (2) 

Homogeneous 

Robots 

Robot-Task-Target  Pairing 

Sim  (3) 

Heterogeneous 

Robots 

Robot-Task-Target  Pairing 

(a)  Master  subsystem 


(b)  Slave  subsystems  (leader/followers) 


FIGURE  1 :  Modified  SMMS  Systems 

force  acting  against  the  target  to  make  a  firm  grip.  Uf,  Uj,  U0, 
and  Ce  were  proposed  in  [2] .  xm  and  xsj  are  the  master  and  the  ith 
slave  robot  position  vectors,  respectively.  xscn  is  the  reference  po¬ 
sition  vector  of  the  ith  slave  robot.  Mm  is  the  inertia  matrix  of  the 
master  robot.  Km  is  the  control  parameters  for  the  linear  diagonal 
master  matrices.  MSi  is  the  inertia  matrices  of  the  ith  slave  robots. 
Bsi  is  the  slave  impedance  matrix.  Ksi  is  the  control  parameters 
for  the  linear  diagonal  slave  matrices,  a  and  A  are  the  control 
parameters  of  the  Ith  slave  robot.  When  the  robot  is  selected  as 
a  team  leader,  a  is  turned  into  one;  otherwise,  it  becomes  zero. 
When  the  robot  is  appointed  as  a  subteam  leader,  A  becomes 
one;  otherwise,  A  is  zero.  Bm  is  the  master  adaptive  impedance 


matrix.  esi  =  xsi  -  (ay!m  +  (1  -  a)X!deal)(al  +  (1  -  a,\) ypos). 
em  =  xm  —  Xh,  where  x^  is  the  position  vectors  commanded  by 
the  human  operator.  xsi  is  the  slave  current  robot  positions.  CC\ 
=  \sgn(e2si)  |,  which  is  the  constant  positive  integers  switching  be¬ 
tween  zero  and  one  in  order  to  determine  the  output  of  the  target 
matrix.  y/pos  is  the  matrix,  [o  o  i]r  to  produce  its  reference  po¬ 
sition  vectors  transformed  from  X[deal  x'm  and  x'si  are  the  delayed 
transmitted  xm  and  xsi,  respectively.  Xicieai  is  the  slave  subteam 
robot  reference  position  vectors.  8Fsi  =  Fsi  —  Ficieai  ( 1  —  oq )  y force 
is  a  difference  between  reference  and  measured  forces  of  the 
slave  robots  when  Ficieai  is  the  reference  force  vectors  and  Fsi 
is  the  measured  forces  of  the  slave  robots,  y force  is  the  ma¬ 
trix,  [o  1  o]r  to  produce  its  reference  force  vectors  transformed 
from  F[deal  The  slave  team  leader  is  remotely  controlled  by  the 
human  operator  to  guide  the  team.  As  mentioned  above,  when 
the  team  is  approaching  the  targets,  it  is  autonomously  split  into 
subteams  paired  to  tasks  and  targets  with  the  robot-task-target 
pairing  method  by  solving  Eqs  (6)  within  the  constraint,  Eq. 
(7).  Robot- task-target  matrices  are  generated  using  Eqs  (6)  and 
transformed  into  the  reference  positions  and  forces  for  the  robot 
to  accomplish  the  assigned  tasks  on  the  assigned  targets.  Fur¬ 
thermore,  during  navigation  to  the  assigned  target,  the  subteam 
leader-follower  formation  is  maintained  or  distorted  by  integrat¬ 
ing  (1)  the  virtual  robot-robot  bondings  with  different  strengths 
based  on  which  two  team  robots  are  connected,  (2)  the  attraction 
to  the  target  with  regard  to  robot-target  distances,  and  (3)  the 
repulsion  from  the  obstacles  with  regard  to  robot-obstacle  dis¬ 
tances.  In  such  a  formation,  all  followers  in  the  sub  team  move 
with  regard  to  the  subteam  leader’s  motion.  After  the  target  is 
reached,  the  slave  robots  will  perform  the  assigned  tasks,  such  as 
target  capture  or  transportation  relying  on  the  robot-task-target 
matrices.  For  target  transportation,  the  contact  force  acting  on 
the  target  by  each  subteam  robot  is  adjusted,  which  could  cause 
the  subteam  robots  to  have  a  firm  grip  of  the  target. 

SIMULATION  RESULTS 

In  simulations,  Sim  (1)  -  Sim  (3),  as  shown  in  Table  2,  the 
SMMS  control  methods  with  and  without  the  robot-task-target 
pairing  method  for  homogeneous  and  heterogeneous  robots  were 
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simulated  in  the  presence  of  time- varying  communication  delays 
to  generate  results  for  performance  improvement.  The  simulated 
communication  delay  varied  from  0  to  0.1  seconds  randomly. 
The  maximum  communication  delay  of  0. 1  second  was  chosen  in 
the  simulations  because  for  the  earth  application,  there  is  a  criti¬ 
cal  value,  beyond  which  the  system  tends  to  become  unstable  [2]. 
In  the  simulations,  as  shown  in  Figure  2(a),  a  master  robot  was  a 
joystick  connected  to  a  laptop  that  read  human  operator  motion 
commands  and  sent  human  commands  to  a  virtual  slave  robot 
model  .  The  virtual  slave  robot  models  in  Figure  2(b)  were  pro¬ 
grammed  to  execute  the  transmitted  commands  and/or  generate 
and  follow  reference  positions  and  velocities  to  perform  the  as¬ 
signed  tasks  on  the  targets.  In  Sim  (1)  and  (2),  as  shown  in 
Figure  4,  seven  robots  were  simulated  as  holonomic  mobile  plat¬ 
forms,  all  of  which  has  two  active  wheels,  with  a  manipulator 
atop  to  form  a  team.  In  Sim  (3),  four  of  the  team  robots  were 
with  manipulators  atop  as  shown  in  Figure  4  when  the  others 
were  without  manipulators  atop  as  shown  in  Figure  3.  Moreover, 
six  virtual  static  obstacles  and  two  virtual  targets  were  modeled 
as  mass-spring-damper  systems  [2].  All  virtual  obstacle,  target, 
and  robot  positions  and  velocities  were  assumed  to  be  known  in 
the  simulations.  The  two  simple  tasks,  transportation  and  cap¬ 
ture,  were  performed  by  the  slave  robots  simultaneously.  TB 
was  transported  by  at  least  three  mobile  robots  when  TA  was 
also  captured  by  at  least  three  mobile  robots.  TB  was  placed  on 
a  movable  platform  with  four  passive  omni-directional  wheels 
tightly  touching  the  ground.  There  was  no  slip  between  the  sur¬ 
faces  of  the  ground  and  the  wheels.  Besides,  TA  was  fixed  on 
the  ground.  It  was  captured  while  being  encircled  by  the  three 
mobile  robots. 

The  simulations  were  set  up  with  the  following  parameters. 
The  desired  safety  distance  between  two  robots  was  set  to  3m. 
The  minimum  distance  between  a  robot  and  an  obstacle  was  set 


(a)  Master  Subsystem 


(b)  Slave  Subsystem 


FIGURE  2:  SMMS  Simulation  Setup 


FIGURE  3:  Mobile  platform 


FIGURE  4:  Mobile  platform  with  the  arm 


to  5m.  Six  circular  objects  with  the  radii  of  5m  were  used  as 
obstacles  in  each  simulation.  In  the  simulations,  the  six  circular 
obstacles,  Obl-6,  were  situated  at  (30,  60),  (50,  40),  (70,  20), 
(70,-20),  (50,-40),  and  (30,-60),  respectively.  Another  two  circu¬ 
lar  objects  with  the  radii  of  5m  represented  targets,  TA  and  TB, 
in  each  simulation.  TA  and  TB  were  initially  static  and  situated 
at  (90,  30)  and  (90,  -30),  respectively,  as  shown  in  Figures  5  and 
9.  The  seven  slave  robots,  Rl-7,  were  initially  located  at  (0,  15), 
(0,  10),  (0,  5),  (0,  0),  (0,  -5),  (0,  -10),  and  (0,  -15),  respectively. 
Only  two  directions  parallel  to  the  ground  were  considered  in 
the  simulations.  Each  slave  robot  was  represented  by  a  circular 
object  with  the  radius  of  3m  in  simulations.  The  slave  robots 
with  transporting  TB  were  commanded  to  move  from  (90,  -30) 
to  (130,  -30)  in  Figure.  In  the  simulations,  the  following  param¬ 
eters  were  used: 

Mm  =  3  kg,  Km  =  6  Ns/m,  Msi  =  30  kg,  Bsi  =  1.0  Ns/m,  Ksi  =  60 
N/m,  /mu  =  10,  ke  =  100,  be  =  60,  =  5,  rsmin  =  5,  kf  -  1,  a 

=  p  =  1,  ft  =  10000,  j3o  =  500,  (j)  =  100,  and  A,-  =  (p  =  y  =  yw  =1 

Simulation  -  Sim(1) 

In  Sim  (1),  the  seven  robots  formed  a  team  teleoperated  by 
a  human  operator  via  the  master  robot.  The  human  operator  re¬ 
motely  controlled  the  team  leader,  R4,  to  reach  TA,  and  all  other 
slave  robots,  Rl-3  and  R5-7,  were  coordinated  with  the  team 
leader  to  surround  TA  to  capture  it.  After  the  TA  capture,  the 
human  operator  commanded  the  team  leader,  R4,  to  move  to  TB 
while  other  robots,  Rl-3  and  R5-7,  were  also  moving  with  regard 
to  the  team  leader  motion  to  approach  TB.  During  the  team  nav¬ 
igations  to  catch  TA  and  TB  in  Figure  5,  all  robots  in  the  team 
were  able  to  avoid  the  obstacles,  Obl-6  while  the  robots  kept  a 
constant  distance  from  each  other.  As  long  as  the  team  leader, 
R4,  telecontrolled  by  the  human  operator  had  a  contact  with  TB, 
the  other  robots,  Rl-3  and  R5-7,  encircled  and  contacted  with  it, 
and  then  all  robots,  Rl-7,  induced  and  regulated  contact  forces 
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FIGURE  5:  Sim  (1)  -  Actual  Path  Trajectories 


FIGURE  6:  Sim  (1)  -  Slave  Forces 
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FIGURE  7:  Sim  (1)  -  Slave  Position  Errors 
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FIGURE  8:  Sim  (1)  -  Slave  Force  Errors 


against  it  as  shown  in  Figure  6  in  order  to  have  a  firm  grip  of  it. 
In  Figure  5,  TB  was  transported  in  40(m)  from  (90,  -30)  to  (130, 
-30),  and  Sim  (1)  was  finished  in  1760  seconds. 

In  Figures  6  and  8,  the  contact  forces  were  maintained  at 
10  (N),  when  force  errors  8Fsi  varied  between  1.0  and  0.0  (N), 
and  its  average  was  0.65  (N),  which  was  acceptable  as  mentioned 
above.  The  force  errors  were  a  little  high  due  to  the  communica¬ 
tion  delays  between  the  robots. 

In  Figures  7,  position  errors  esi  of  the  team  leader  and  Rl- 
6  were  presented,  respectively.  The  position  errors  varied  from 
2.5  to  0  (m),  which  was  mostly  caused  by  the  time-varying  com¬ 
munication  delays.  The  position  error  average,  0.65  (m),  was 
still  acceptable  because  the  team  leader  robot  teleoperated  by  the 
human  operator  moved  slowly  when  the  other  robots,  Rl-3  and 
R5-6,  moved  with  regard  to  the  team  leader  positions. 
Simulation  -  Sim(2) 

In  Sim  (2),  two  tasks,  Task  1,  i.e.  target  transporting  and 
Task  2,  i.e.  target  capturing,  were  performed.  The  seven  robots, 
Rl-7,  could  form  35  types  of  Robot  Combinations  (Subteams 
(Subl-35))  as  shown  in  Table  3. 


For  Task  1,  the  desired  contact  forces  were  8.0(N),  and  the 
target,  TB,  was  moved  from  (90,-30,0)  to  (130,-30,0).  For 
Task  2,  the  desired  contact  forces  were  0(N),  and  the  target,  TA, 
was  not  moved  because  the  task  did  not  require  the  robot  subteam 
to  carry  the  target.  With  the  robot-task-target  pairing  method 
mentioned  in  Eqs  (1)  -  (5),  the  WAGT  Tables  were  generated. 
Subteams  (Subl  -  35)  and  their  bids  for  Task  1  {t\)  and  Task 
2  fe)  were  found  for  TA  and  TB  in  Table  4.  Bids  in  Table  4, 
(Ta,Tb)  where  Ta  is  the  bid  value  for  TA  when  Tb  is  for  TB, 
were  calculated  in  Eq.  (5)  as  an  inverse  of  the  sum  of  target-robot 
distances  in  a  subteam  minus  the  base  price  when  the  base  price 
for  t\  was  30  and  ^  was  10.  The  reasons  were  that  in  order  to 
start  with  the  tasks,  the  robots  needed  to  maintain  at  least  30(m) 
from  TB  for  t\  when  only  keeping  at  least  10(m)  from  TA  for 
because  the  robots  need  more  space  to  do  t\  than  t^. 

As  shown  in  Figure  9,  only  the  team  leader,  R4,  was  teleop¬ 
erated  by  the  human  operator  when  all  other  robots,  Rl-3  and  R5- 
7,  automatically  formed  two  subteams,  (Rl-3  and  R5-7  combi¬ 
nations)  to  capture  TA  and  transport  TB  simultaneously  in  1250 
seconds,  respectively.  R4  was  not  engaged  in  any  task,  which 
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TABLE  3:  Robot  Combinations  (Robot  Subteams) 


Subteam 

Combos 

Subteam 

Combos 

Subteam 

Combos 

Subl 

R1  R2  R3 

Sub  13 

R1  R5  R6 

Sub25 

R2  R6  R7 

Sub2 

R1  R2  R4 

Sub  14 

R1  R5  R7 

Sub26 

R3  R4  R5 

Sub3 

R1  R2  R5 

Sub  15 

R1  R6  R7 

Sub27 

R3  R4  R6 

Sub4 

R1  R2  R6 

Sub  16 

R2  R3  R4 

Sub28 

R3  R4  R7 

Sub  10 

R1  R4  R5 

Sub22 

R2  R4  R7 

Sub34 

R4  R6  R7 

Sub  11 

R1  R4  R6 

Sub23 

R2  R5  R6 

Sub  3  5 

R5  R6  R7 

Sub  12 

R1  R4  R7 

Sub24 

R2  R5  R7 

S  »  *0  GO  SO  1W  120  140 

K  pMtfons(M) 


FIGURE  9:  Sim  (2)  -  Actual  Path  Trajectories 


TABLE  4:  Weighted  Attack  Guidance  Table  (WAGT)  for  Target 
A  and  B 


Subteam 

Bids 

Subteam 

Bids 

Subteam 

Bids 

Subl 

(41,69) 

Sub  13 

(39,73) 

Sub25 

(38,76) 

Sub2 

(40,69) 

Sub  14 

(39,74) 

Sub26 

(39,74) 

Sub3 

(40,70) 

Sub  15 

(38,75) 

Sub27 

(39,75) 

Sub4 

(40,71) 

Sub  16 

(40,71) 

Sub28 

(39,75) 

Sub  10 

(40,72) 

Sub22 

(39,74) 

Sub34 

(38,78) 

Sub  11 

(39,73) 

Sub23 

(39,75) 

Sub35 

(38,79) 

Sub  12 

(39,73) 

Sub24 

(39,75) 

could  reduce  the  time  delay  effect  on  the  task  achievements.  All 
tasks  were  done  by  the  two  subteams,  Subl  and  Sub35,  fully  au¬ 
tonomously.  In  Figure  10,  the  simulation  results  showed  that  the 
contact  forces  were  maintained  at  10  (N)  when  the  force  errors, 
8Fsi ,  varied  from  0.0  to  0.9(N),  and  force  error  average  was  0.45 
(N)  The  position  errors,  esu  varied  from  0  to  0.12  (m)  in  Fig¬ 
ures  11,  and  a  position  error  average  was  0.05  (m).  The  8Fsi  and 
esi  were  caused  by  robotic  path  adaptation  due  to  the  modified 
potential  field  based  formation  control  method  and  time-varying 
communication  delays  between  the  robots.  By  comparing  those 
errors  in  Figures  11  and  12  and  7  and  8,  the  performance  of  the 
system  in  Sim  (2)  was  better  than  that  in  Sim  (1).  The  tasks  were 
finished  more  quickly  in  1250  seconds,  and  the  position  and  force 
errors  were  smaller  in  Sim  (2)  for  two  reasons.  (1)  the  amount  of 
information  transmitted  over  the  time-varying  links  between  the 
master  and  slave  subsystems  became  less  in  Sim(2)  than  Sim(l) 
when  only  autonomous  local  slave  robots  handled  the  tasks,  but 
the  teleoperated  R4  acted  as  a  supervisor  to  monitor  other  robot 
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FIGURE  10:  Sim  (2)  -  Slave  Forces 


operations.  (2)  Forming  the  subteams  could  save  all  seven  robots 
from  visiting  all  targets  to  complete  two  tasks.  The  seven  robots 
were  split  into  three  robots  in  one  subteam  to  perform  the  task 
on  each  target  simultaneously  as  shown  in  Figure  9.  By  taking 
advantage  of  the  task  planning  independently  done  by  each  sub¬ 
team,  the  task  completion  effectiveness  was  enhanced  when  the 
operation  time  was  decreased  to  1250  seconds  in  Sim(2)  from 
1760  seconds  in  Sim(l)  in  Figures  1 1  and  7  as  the  robot  average 
speed  was  constant  during  the  simulations. 

Simulation  -  Sim(3) 

In  Sim  (3),  Rl-3  were  shown  in  Figure  4  when  R4-7  were 
shown  in  Figure  3.  The  obstacles,  targets,  and  tasks  were  equiva¬ 
lent  to  the  ones  specified  in  Sim  (1-2).  By  solving  Eqs.  (6)  within 
Eq.  (7),  Table  5  was  generated.  Therefore,  Sub35  was  paired  to 
TA  for  the  task  of  the  target  capture  when  Subl  was  paired  to 
TB  for  the  task  of  the  target  transportation,  and  R4  was  a  team 
leader. 

In  Figure  14,  the  simulation  results  showed  that  the  contact 
forces  were  also  maintained  at  10  (N),  which  represented  a  firm 
grip  of  TB.  Moreover,  in  Figure  16,  the  force  errors  varied  from 
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FIGURE  1 1 :  Sim  (2)  -  Slave  Position  Errors 


TABLE  5:  Weighted  Attack  Guidance  Table  (WAGT)  for  Target 
A  and  B  in  Sim(3) 


Subteam 

Bids 

Subteam 

Bids 

Subteam 

Bids 

Subl 

(41,369) 

Sub  13 

(239,173) 

Sub25 

(238,176) 

Sub2 

(140,269) 

Sub  14 

(239,174) 

Sub26 

(239,174) 

Sub3 

(140,270) 

Sub  15 

(238,175) 

Sub27 

(239,175) 

Sub4 

(140,271) 

Sub  16 

(240,171) 

Sub28 

(240,175) 

Sub  10 

(240,172) 

Sub22 

(239,174) 

Sub34 

(328,178) 

Sub  11 

(239,173) 

Sub23 

(239,175) 

Sub35 

(338,179) 

Sub  12 

(239,173) 

Sub24 

(239,175) 

FIGURE  12:  Sim  (2)  -  Slave  Force  Errors 


FIGURE  13:  Sim  (3)  -  Actual  Path  Trajectories 


0.0  to  0.85(N),  and  their  average  was  0.25  (N)  when  in  Figures 
15,  the  position  errors  were  recorded  from  0  to  0.35  (m),  and 
their  average  was  0.08  (m).  By  comparing  the  results  in  Sim(2) 
and  Sim(3),  their  recorded  force  and  position  errors  were  similar, 
and  their  mission  completion  time  was  not  quite  different.  There¬ 
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fore,  the  performance  of  the  proposed  system  was  not  affected  by 
using  the  heterogeneous  robots. 

CONCLUSION  &  FUTURE  WORK 

The  control  method  integrating  the  above  mentioned  main 
components  is  developed  for  the  SMMS  teleoperations  to  do  the 
multi-task  on  the  multi-target  and  improve  the  performance  in 
terms  of  the  effectiveness  of  the  task  achievement.  Nonetheless, 
the  proposed  robot-task-target  pairing  method  could  generate  a 
suboptimal  solution  in  general  since  it  is  heuristic. 

Therefore,  our  future  work  will  be  to  further  evaluate  the 
performance  of  using  the  proposed  robot-task-target  pairing 
method  to  verify  the  performance  and  quality  of  the  pair  solu¬ 
tions.  In  addition,  we  will  look  into  the  proposed  control  method 
to  team  heterogeneous  robots  working  in  much  complicated  tasks 
and  environments,  e.g.  an  uncertain  task  that  may  include  uncon¬ 
strained,  constrained,  transition,  or  some  motions  combining  two 
or  all  of  them  in  an  unknown  area,  which  has  not  been  seen  in  this 
paper. 
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FIGURE  15:  Sim  (3)  -  Slave  Position  Errors 
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FIGURE  16:  Sim  (3)  -  Slave  Force  Errors 
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